REMARKS 

Claims 1-44 are in this application and are presented for consideration. By this 
amendment, Applicant has amended claims 1 and 23. Applicant has canceled claims 2 and 24. 
The features of claims 2 and 24 have been placed into claims 1 and 23 respectively. It is 
Applicant's position that no new issues are presented. 

Claim 1 has been rejected as obvious in view ofWatanabe et al. (US 6,763,284) in view 
of Huissoon (US 6,044,308) and Roos (6,615,1 12). 

Claim 1 relates to a method for fading computer-generated information into an image 
of the real environment detected by an image receiving unit located on a viewing device. The 
method comprises the steps of providing the viewing device with the image receiving unit. The 
image of the real environment is then detected by the image receiving unit. Computer- 
generated information is then faded into the image. The position and orientation or pose of the 
image receiving unit is determined and the robot-specific information corresponding to this 
determination is faded over the image of the real environment on the viewing device. At least 
one robot-specific coordinate system is faded into the image. The present invention permits 
greater flexibility of the automation of operating sequences by means of robots because 
reprogramming is facilitated. The present invention makes putting into operation, programming 
and operating of robots more simple and efficient. The manipulatability of the reference 
coordinate systems is advantageously cost efficient since it makes it unnecessary for the user 
to learn and carry out complicated measurement methods. The prior art fails to provide such 
features or advantages. 
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Applicant would like to note that fading in has a meaning to one of ordinary skill in the 
art. This term is a term of art commonly used in the computer and video arts. A person of 
ordinary skill in the art would understand the use of such a term. As such, Applicant is not 
required to give a special meaning to the term "fading in". In the absence of a specific definition 
of a term, the ordinary meaning should be used. 

Watanabe et al. discloses a robot teaching apparatus. Mounted at an arm distal end of 
a robot 5 are a camera 2 and a projector 1 . The projector 1 projects a pattern of light. A robot 
controller 6 houses an image processing device therein and it is connected to the robot 5, the 
projector 1, the image pickup device 2, a display device 3 and a teaching operation panel (not 
shown). The robot 5 is moved with a manual operation to a position and an attitude at which 
it is properly spaced from a reference work 7 and it looks over the entire working space. An 
image including the reference work 7 in a field of view is picked up by the camera and the 
image is displayed on the image display device 3. A 2-dimensional image position pointing 
device 4 utilizing a mouse is provided in the image display device 3 and an operator points to 
a position (uO, vO) to be measured by the robot 5 on the image using the pointing device 4. The 
pointed position (uO, vO) is stored in the robot controller 6. The pointing device 4 is moved 
along a plurality of points to constitute a working line on an image. After the measurement 
starting point is on the working line, a view line corresponding to the position (uO, vO) is 
obtained using the position (uO, vO) on the 2-dimensional image and the position and orientation 
of the image pickup device 2. A slit light is projected from the projector 1 onto the reference 
work 7 and measurement of a 3-dimensional position on the reference work 7 is started. An 
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image, including an image of the bright line formed on the reference work 7, is photographed 
by the camber 2. The robot is moved along a working path a small distance and light projecting 
and photographing are performed for each small distance so that 3-dimensional position points 
at which the bright line crosses the working line are sequentially obtained. The detection results 
are verified by superimposing the working line on to the image of the camera 2 on the screen 
of the image display device 3. 

Huissoon discloses a method and device for robot tool frame calibration. A robot 30 
comprises a base 32 and a manipulator 34. The manipulator 34 is provided with several 
linkages 36 connected to joints 38 with an end effector 40. A tool having a tool center point 
(TCP) is attached to end-point 40 of robot 30 and is robotically controlled by the robot. The 
pose of coordinate frame E of the robot end-point 40 with respect to a global coordinate 
reference frame G of robot 30 is defined by a forward kinematic model of the robot so that 
given the angels of joints 38 and the lengths of links 36 and the robot geometry, the pose of 
frame E can be computed with respect to frame G. A reference fixture 44 is mounted on a 
calibration jig 60 comprising a support table 62 and one or more active area sensors 64 are 
attached to the support in known positions with respect to the reference fixture 44. The sensors 
64 are located below the viewing window 66 located in table 62. A laser grid generator 63 
illuminates the edges of a gas cup and the tip of a contact tube so that the analysis area sensor 
64 image data can establish the location of TCP 42 with respect to reference fixture 44. The 
TCP is the focal point of the welding laser and its position is determined by analyzing the image 
of the focusing laser spot on a projection screen 66. Since the TCP sensor is in a known 
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position with respect to the structured light sensor calibration features, the TCP position with 
respect to the sensor can be computed. 

Roos discloses a method and device for multistage calibration of multiple-axis 
measuring robots 6 and associated optical measuring devices 10 in a measuring station 1 for a 
workpiece 2. Calibration occurs in ameasuring cascade comprising three calibrating steps. The 
optical measuring device 10 and the operating point 28 thereof, the manipulator 6 and the axes 
thereof and the allocation of the manipulator 6 with respect to the workpiece 2 are successively 
calibrated. 

Watanabe et al. fails to teach or disclose fading in coordinate systems into an image of 
the real environment. At most, Watanabe et al. teaches measuring the working line of a 
component by measuring sensors and automatically teaching the path thus measured. The path 
of Watanabe et al. is superimposed onto the image of the workpiece and the user is asked to 
set the points of the path manually to verify the detection results. The present invention takes 
a different approach. The present invention provides a method for displaying abstract 
information that is not visible to the user so that the user has a visual representation of the 
robot-specific conditions actually occurring. The present invention fades in a system of 
coordinates over a real environment image. In contrast, the Watanabe disclosure suggests a 
system of coordinates that are transformed into another system of coordinates. Watanabe et 
al. merely teaches using calculations to transform one system of coordinates to another system 
of coordinates to compare the actual working line path with the set working line path. In the 
present invention, the coordinate system is faded into the image so that the user has a visual 
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picture of the robot-specific conditions occurring. As such, the prior art teaches a different 
approach and does not suggest the features of the present invention. 

The Huissoon reference fails to teach or suggest the combination of fading computer- 
generated information into a real environment image. At most, Huissoon teaches a laser grid 
generator 63 that illuminates the edges of a gas cup and the tip of a contact tube so that the 
analysis area sensor 64 image data can establish the location of the tool center point 42 with 
respect to reference fixture 44. The present invention takes a different approach. The present 
invention determines the position of the image receiving unit. The robot-specific information 
that results from that determination is laded onto the real environment image. In contrast, 
Huissoon merely provides for a kinematic model to determine the pose of frame E with respect 
to frame G. In fact, Huissoon fails all together to disclose a viewing device with the image 
receiving unit. As such, Huissoon fails to provide any motivation for the features found in the 
present invention. 

Similarly, Roos fails to provide any motivation for fading computer-generated 
information into a real environment image. At most Roos discloses successively calibrating the 
optical measuring device 10 and the operating point 28 thereof, the manipulator 6 and the axes 
thereof and the allocation of the manipulator 6 with respect to the workpiece 2. Roos fails to 
mention anything about providing a viewing device with an image receiving unit. In contrast, 
the present invention has a viewing device with an image receiving unit. The position of the 
image receiving unit is determined and the robot-specific information provided by this 
determination is faded into the real environment image. This is a completely different approach 
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compared to Roos. In fact, Roos fails to suggest providing any type of real environment image. 
As such, Roos suggests a different approach and fails to teach the features of the present 
invention. Accordingly, Applicant respectfully requests that claim 1 be favorably considered 
as now amended and all claims that depend thereon. 

Claims 23-27, 39 and 41-44 have been rejected under 35 U.S.C. 102(e) as being 
anticipated by Watanabe et al. 

Watanabe et al. fails to teach or disclose fading in coordinate systems into an image of 
the real environment. At most, Watanabe et al. teaches measuring the working line of a 
component by measuring sensors and automatically teaching the path thus measured. The path 
of Watanabe et al. is superimposed onto the image of the workpiece and the user is asked to 
set the points of the path manually to verify the detection results. The present invention takes 
a different approach. The present provides a method for displaying abstract information that 
is not visible to the user so that the user has a visual representation of the robot-specific 
conditions actually occurring. The present invention fades in a system of coordinates over a 
real environment image. In contrast, the Watanabe disclosure suggests a system of coordinates 
that are transformed into another system of coordinates. Watanabe et al. merely teaches using 
calculations to transform one system of coordinates to another system of coordinates. In the 
present invention, the coordinate system is faded into the image so that the user has a visual 
picture of the robot-specific conditions occurring. As such, the prior art teaches a different 
approach and does not suggest the features of the present invention. Accordingly, Applicant 
respectfully requests that the Examiner favorably consider Claim 23 and all claims that depend 
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thereon. 

The prior art as a whole fails to direct the person of ordinary skill in the art toward the 
feature of the invention. Further, the invention includes cooperating features which provide 
particular advantages which are neither taught nor suggested by the prior art. Accordingly, 
Applicant requests that the Examiner favorably consider the amended claims in light of the 
discussion above. 

Further and favorable consideration on the merits is requested. 



Respectfully submitted 
For Applicant, 
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John James McGlew 
Reg. No. 31,903 
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